Chief Technology Officers (CTOs), Chief Information Officers (CIOs), and other responsible personnel are frequently tasked with decisions related to the efficiency of infrastructure management. In large organizations, a CTO may be responsible for managing infrastructure within each business unit or other operating environment. Particular choices of hardware and software systems, combinations thereof, and the placement of these systems within a business impact resource efficiency. Thus, CTOs or CIOs for various business units or corporations have a need to gauge cost and performance parameters for infrastructure configuration and infrastructure usage based on implemented decisions.
Traditionally, decision making processes of CTOs have been painstaking, requiring many man-hours to perform calculations related to determining the approximate performance values associated with existing servers, and then determining where to make strategic purchases for new servers and platforms. This problem has existed because the performance of the same component may vary depending upon how that component is implemented in a server, or based upon the operating system or platform the component operates in.
Currently, various benchmarking techniques are available to assess relative performance with respect to computer hardware and computer software. Hardware benchmarking tools may, for example, assess the floating point operation performance of a CPU, and software benchmarks may, for example, run against compilers or database management systems.
Over the years, various measures have been available for different computing performance parameters. Million instructions per second (MIPS) was developed as a standard measure of processing speed. However, MIPS measurements have been found to be heavily dependent on the programming language used and furthermore are not comparable between architectures. Additionally, MIPS measurements are only applicable to mainframes.
Similarly to MIPS, floating point operations per second (FLOPS) also provide a measure of a computer's performance. However, there are many factors in computer performance other than raw floating-point computation speed, such as I/O performance, interprocessor communication, cache coherence, and the memory hierarchy. Furthermore, FLOPS measurements are applicable only to single computing devices.
Benchmarks provide a method of comparing the performance of various subsystems across different chip/system architectures. Types of benchmarking tools may include for example real programs, kernels, component or micro-benchmarks, synthetic benchmarks, parallel benchmarks. Some known benchmarking tools are offered by Business Applications Performance Corporation (BAPCo), Embedded Microprocessor Benchmark Consortium (EEMBC), Standard Performance Evaluation Corporation (SPEC), and Transaction Processing Performance Council (TPC).
Benchmarking tools typically offer the ability to gauge for example, the efficiency of running a program on each of multiple different types of servers or computers. After running the program on a server or computer, the benchmarking tool will typically provider a number for each server or computer that operates as in indicator of the efficiency of that particular server or computer for running the program.
While benchmarking is a useful tool, it fails to provide a single standard for measuring efficiency within an organization across multiple devices and applications and further fails to provide an efficiency indicator based on actual implementation factors. Additional complexities arise for example when virtualization is implemented, or when a single device is shared for performance of multiple tasks. A solution is needed for providing a normalized measurement system so that processing performance can be measured and improved where necessary.
With improved knowledge of infrastructure configuration and usage costs and the fluctuations in these costs, responsible parties would be able to assess cost and performance impacts of actions such as buying smaller and cheaper servers, reducing the number of applications running on a server, using lower cost CPUs, etc. However, no standard measure is available for gauging these costs and fluctuations across disparate environments incorporating disparate technologies. Accordingly, a solution is needed for normalizing the measurements across disparate technologies.
A solution is further needed in the industry for evaluating performance capacity of various computing platforms in relation to the cost of those platforms, to facilitate that managerial and budgeting decision-making. Accordingly, a methodology is needed that allows comparison across a multiple of different platforms from an infrastructure perspective so different platforms, different CPUs, different CPU cycle speed, different server pipes, etc. are evaluated on an equivalent level in terms of cost and performance.